Papers with iterative offline RL training
Preference-Guided Reflective Sampling for Aligning Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Repeated random sampling is a widely used method that independently queries the model multiple times to generate outputs. |
| Approach: | They propose a more efficient method for iterative data generation and model re-training that leverages tree-based tree-derived generation framework to enable more efficient sampling. |
| Outcome: | The proposed method significantly outperforms repeated random sampling in best-of-N sampling on AlpacaEval and Arena-Hard. |